Papers with student’s learning process
Teaching-Assistant-in-the-Loop: Improving Knowledge Distillation from Imperfect Teacher Models in Low-Budget Scenarios (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated state-of-the-art (SOTA) performance across a wide spectrum of tasks. |
| Approach: | They propose a framework that leverages three signal types to improve efficiency within resource-constrained, imperfect teacher scenarios. |
| Outcome: | The proposed framework improves on four complex reasoning tasks by 20.79% compared to fine-tuning without any signals across datasets. |